A Modular Software Framework for Compression of Structured Climate Data

Through the introduction of next-generation models the climate sciences have experienced a breakthrough in high-resolution simulations. In the past, the bottleneck was the numerical complexity of the models, nowadays it is the required storage space for the model output. One way to tackle the data storage challenge is through data compression. In this article we introduce a modular framework for the compression of structured climate data. Our modular framework supports the creation of individual predictors, which can be customised and adjusted to the data at hand. We provide a framework for creating interfaces and customising components, which are building blocks of individualised compression modules that are optimised for particular applications. Furthermore, the framework provides additional features such as the execution of benchmarks and validity tests for sequential as well as parallel execution of compression algorithms

Adaptive Lossy Compression of Complex Environmental Indices Using Seasonal Auto-Regressive Integrated Moving Average Models

Significant increases in computational resources have enabled the development of more complex and spatially better resolved weather and climate models. As a result the amount of output generated by data assimilation systems and by weather and climate simulations is rapidly increasing e.g. due to higher spatial resolution, more realisations and higher frequency data. However, while compute performance has increased significantly because of better scaling program code and increasing number of cores the storage capacity is only increasing slowly. One way to tackle the data storage problem is data compression. Here, we build the groundwork for an environmental data compressor by improving compression for established weather and climate indices like El Niño Southern Oscillation (ENSO), North Atlantic Oscillation (NAO) and Quasi-Biennial Oscillation (QBO). We investigate options for compressing these indices by using a statistical method based on the Auto Regressive Integrated Moving Average (ARIMA) model. The introduced adaptive approach shows that it is possible to improve accuracy of lossily compressed data by applying an adaptive compression method which preserves selected data with higher precision. Our analysis reveals no potential for lossless compression of these indices. However, as the ARIMA model is able to capture all relevant temporal variability, lossless compression is not necessary and lossy compression is acceptable. The reconstruction based on the lossily compressed data can reproduce the chosen indices to such a high degree that statistically relevant information needed for describing climate dynamics is preserved. The performance of the (seasonal) ARIMA model was tested with daily and monthly indices

Ozone assessment as an EOSC-Synergy thematic service

Ozone assessment is an important task for Climate and Environment studies. The ozone assessment service (O3as) project is going to support scientists and everyone interested in determining ozone trends for different parts of the world. It is one of the thematic services of the EOSC-Synergy project. The service applies a unified approach to analyse results from a large number of different chemistry-climate models, helps to harmonise the calculation of ozone trends efficiently and consistently, and produce publication-quality figures in a coherent and user-friendly way. Among other tasks it will aid scientists to prepare the quadrennial Global Assessment of Ozone depletion. It will also allow access to the high-level data by citizens. The service relies on several containerized components distributed across the cloud (Kubernetes) and HPC resources and leverages large scale data facility (LSDF)

An online service for analysing ozone trends within EOSC-synergy

The European Open Science Cloud-Synergy (EOSC-Synergy) project delivers services that serve to expand the use of EOSC. One of these services, O3as, is being developed for scientists using chemistry-climate models to determine time series and eventually ozone trends for potential use in the quadrennial Global Assessment of Ozone Depletion, which will be published in 2022. A unified approach from a service like ours, which analyses results from a large number of different climate models, helps to harmonise the calculation of ozone trends efficiently and consistently. With O3as, publication-quality figures can be reproduced quickly and in a coherent way. This is done via a web application where users configure their queries to perform simple analyses. These queries are passed to the O3as service via an O3as REST API call. There, the O3as service processes the query and accesses the reduced dataset. To create a reduced dataset, regular tasks are executed on a high performance computer (HPC) to copy the primary data and perform data preparation (e.g. data reduction, standardisation and parameter unification). O3as uses EGI check-in (OIDC) to identify users and grant access to certain functionalities of the service, udocker (a tool to run Docker containers in multi-user space without root privileges) to perform data reduction in the HPC environment, and the Universitat Politècnica de València (UPV) Infrastructure Manager to provision service resources (Kubernetes)

5th Data Science Symposium, GEOMAR

Modern digital scientific workflows - often implying Big Data challenges - require data infrastructures and innovative data science methods across disciplines and technologies. Diverse activities within and outside HGF deal with these challenges, on all levels. The series of Data Science Symposia fosters knowledge exchange and collaboration in the Earth and Environment research community. We invited contributions to the overarching topics of data management, data science and data infrastructures. The series of Data Science Symposia is a joint initiative by the three Helmholtz Centers HZG, AWI and GEOMAR Organization: Hela Mehrtens and Daniela Henkel (GEOMAR